Design your own QA system: programme specifications and quality assurance

John Bell, Cambridge University

Presentation at UKCLE seminar on assuring quality in legal education, 11 February 2002

John’s presentation encouraged delegates to take advantage of the processes developed as part of the QAA framework in internal quality assurance processes, focusing on the programme specification.

John is a member of the Quality Assurance Agency’s Law Subject Benchmarking Group and a QAA auditor. He was also involved in the 1999 trials of subject review.

---

The new quality assurance (QA) model for universities is in one sense more trusting of institutions than the previous model, as external inspections will be more limited. However, this is predicated on the assumption that institutions have introduced serious and effective internal processes. Institutions will have to demonstrate and make public the outputs of those processes, rather than having outsiders do this for them.

The pressure is thus on our internal processes. Now is the time to review whether they do the job that we want internally, even before we worry about whether outsiders are happy. Often, on reflection, we will discover the need to align our objectives and practices. We need to be explicit about the criteria by which we judge ourselves to be succeeding and ask ourselves whether our processes are

• coherent do the different activities and testing fit together?
• effective what measures do we use or outsiders set (institutional mission, subject benchmarks, professions, student and parental opinion)?

The programme specification is one tool for examining how we are doing. (For further information on programme specification requirements see
annex D of the Quality Assurance Agency’s Handbook for academic review.

The programme specification

Core Information	Optional Information
1-7 title, awarding body and benchmark group	11 support for learning
8 programme aims	12 admission criteria
9 intended learning outcomes: knowledge and understanding skills and other attributes	13 evaluation and improvement of quality
10 programme structure and content	14 assessment regulations
	15 indicators of programme quality

Programme specification template

How does the programme specification serve the quality assurance needs and interests of a law school? It can be seen as a either document or as a process – for QA, its function as a process is critical.

Much of the information contained in the programme specification is technical, risking making it unreadable. The QAA ambition that a wide range of people will read the programme specification is unlikely to be realised, but it should still be a document which can be used in managing or delivering programmes and serve as the underpinning support for information provided in a more digestible form. The key elements are the links between 8 (aims) and 9 (objectives) and 10 (programme content) and 11 (support for learning). Given 12 (admission criteria) as a background, how does the structure of the programme and available learning support enable the student to achieve the intended outcomes of the programme? Finally 15: what are our measures of success?

The ambition of viewing a programme holistically, rather than in modules or courses, is laudable. Too often we are so busy with our own patch, our own lectures and seminars, that we don’t try to think ourselves into the position of the students who are trying to juggle different subjects at once and to make sense of the different aspects of learning. The programme specification is limited in its ability to achieve this – it focuses on intended links in a production line way and focuses on objectives, processes and outcomes; we design a programme, run modules, put students into them and, hey presto, the students demonstrate certain outcomes in assessment. Of course it is not like that – students learn from many aspects of the learning environment and also suffer from other pressures (home, money, friends), which affect the learning process. At best the programme specification prompts us to think about the connections we have planned, and whether there are gaps.

Mapping student progress

The programme specification fits into a framework of internal QA activities:

• draft the key elements in the programme specification
• conduct a mapping exercise on actual modules
• follow the progress of actual students through the system
• where are the apparent inconsistencies – alter the specification, the modules, the assessment?
• how do we express inputs from institution-wide services and facilities?

The mapping exercise inherent in the process of programme specification is useful here:

• what are the objectives/learning outcomes?
• is it a direct or incidental part of any module?
• is specific instruction or feedback provided?
• where is it specifically practised?
• is it a direct or incidental part of any assessment or recording?
• can we develop a holistic picture of progression and development?

Start with existing statements of aims and objectives and map them onto the different modules, where are they taught, supported or assessed – not every aspect of knowledge or skill forms part of every module. Then track typical students (hypothetical or actual) through the programme and see what turns out to be their learning experience. Are some things over-assessed or over-taught? Do they miss out on some things you think are essential? Are there areas where you have no record of student achievement? Their experience will also trigger you to think about the support they get from elsewhere in the university. Through the map, make sure that you understand how far your provision supports the learning intended, and how you satisfy yourself that all this is happening. Review how well you are doing and where further work is needed.

An illustration of mapping comes from my experience teaching at Leeds. We established our aims in 1992, but in mapping the course for an internal review in 1995 we noted that learning support in parts of the course where oral skills were developed was patchy, as was our recording of how students were doing. The mapping exercise and review led us to be more systematic and to expand the provision to support oral skills.

Students should be able to:	Developed	Assessed
explain an issue clearly and concisely present a clear and logical argument persuade by means of oral argument	contribution to classes voluntary mooting	informal note by tutors (on academic record) assessment criteria in specific modules

Mapping oral skills

Last Modified: 4 June 2010